Convergence Properties of Nonlinear Conjugate Gradient Methods

Recently, important contributions on convergence studies of conjugate gradient methods have been made by Gilbert and Nocedal [6]. They introduce a “sufficient descent condition” to establish global convergence results, whereas this condition is not needed in the convergence analyses of Newton and quasi-Newton methods, [6] hints that the sufficient descent condition, which was enforced by their two-stage line search algorithm, may be crucial for ensuring the global convergence of conjugate gradient methods. This paper shows that the sufficient descent condition is actually not needed in the convergence analyses of conjugate gradient methods. Consequently, convergence results on the Fletcher-Reeves-type and Polak-Ribière-type methods are established in the absence of the sufficient descent condition. To show the differences between the convergence properties of Fletcher-Reevestype and Polak-Ribière-type methods, two examples are constructed, showing that neither the boundedness of the level set nor the restriction βk ≥ 0 can be relaxed for the Polak-Ribière-type methods.